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ABSTRACT 

Conducting observational research in a cross-cultural 
context invokes a unique set" of difficulties. This paper describes 
sources of bias and threats to validity that are specific to 
cross-cultural research. It describes problems of cross-cultural 
research designs for descriptive studies, comparative research, and 
experimental research and outlines techniques for dealing with 
problems. (Contains 12 references.) (LMl) 
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A general overview of bias and validity issues 2 
Abstract 

Sources of bias and threats to validity specific to cross-cultural 
research are examined. The paper is intended to provide a general 
orientation to these issues when evaluating cross-cultural 
research design. Problem areas associated with descriptive sjtudies 
as well as with comparative and experimental research are 
considered. Outlines of techniques and methods useful in dealing 
with the identified areas of difficulty are also suggested. Twelve 
sources are cited primarily from cross-cultural research 
methodology texts and anthologies in the behavioral sciences. 
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Conducting research in a cross-cultural context has its own 
particular set of difficulties associated with it. Higher costs, 
language barriers, inadequate postal and communications services, 
restrictive social norms, political limitations and unforseen 
cultural and social factors can all contribute to the general 
problematic of cross-cultural research. In spite of these 
barriers, scientists from the various social and behavioral 
sciences do engage in cross-cultural studies for a number of 
ethical and personal reasons (Brislin, Lonner & Thorndike, 1973). 

"Any theoretical orientation, or any construct, axiom, 
postulate, or corollary in the social and behavioral sciences can 
be investigated cross-culturally" (Brislin et al., 1973, p. 143). 
Reasons for conducting research in cross-cultural settings include 

(a) the description and categorization of behaviors as- conditioned 
by living in a given country or culture (Brislin et al., 1973), 

(b) comparative studies aimed at increasing the predictive range 
of a hypothesis already demonstrated in the researcher's own 
culture (Lonner & Berry, 1986) and (c) the measuring of beliefs, 
attitudes and values for the implementation or evaluation of 
social programs (Segall, 1986). Although researchers engaged in 
each of these kinds of studies will face similar difficulties in 
conducting studies in a cross-cultural context, there are also 
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methodological problems which are specific to each of these types 

of research in this setting. 

Becoming aware of problem areas particular to cross-cultural 
research is a first step in avoiding the pitfalls inherent in this 
kind of activity. The application of proven cross-cultural 
research methodologies developed by veteran researchers in this 
arena, is another way to insure that studies incorporate certain 
safeguards in their design. The purpose of this paper, then, is to 
identify areas of bias and threats to validity which ^are specific 
to cross-cultural research and to identify techniques or methods 
which will be useful, particularly to the novice, in conducting 
satisfactory cross-cultural studies. 

Observational Methodologies 
Descriptive research has a wide range of cross-cultural 
uses. Perhaps the best know of these applications is the 
ethnographic study, the traditional domain of the cultural 
anthropologist. These studies are characterized by a description 
of group behavior and the institutions of a particular culture or 
society. The majority of these studies are inductive in nature. 
The anthropologist, theoretically speaking, assumes nothing about 
the culture and he/she is there to observe and record the group's 
behavior in its natural environment. Although some ethnographic 
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studies can also be conducted through literature reviews and 
analysis (Lonner 4 Berry, 1986) , the majority of this kind of 
investigation involves field work. 

Field work, however, is not the exclusive domain of 
anthropologists. Political, social and behavioral scientists all 
employ field methods in the collection of data, particularly when 
conducting cross-cultural research. Hunroe 4 Munroe, (1986) 
describe three kinds of information which are sought through 
field-based methodologies. 

1. The collection of basic data such as that obtained 
through conducting a complete census of a community, 
gathering information regarding (a) household organization, 
(b) education, (c) languages spoken, (d) religious 
affiliation, (e) degree of acculturation, (f) occupational 
histories, (g) socioeconomic status, (h) fertility and (g) 
kin relationships. 

2. Ethnographic content for a given cultural group's 
institutions svch as their (a) techno-economic systems, (b) 
social organization, and (c) conceptual framework. 

3. Naturalistic data pertinent to specific research 
variables. This data, such as reputation ratings on member's 
personal and social attributes which are made by a reliable 
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source, can be correlated to social and physiological 
phenomena in. the community. 

While the first of these three types of information 
gathering require survey techniques, the second and third use 
primarily observational procedures. Observational methods are used 
both in descriptive research and in experimental studies. Bochner 
(1986), lists seven specific problem areas inherent in using 
observation as a means of gathering data. These problem areas are 
not exclusive to cross-cultural studies, but this setting does 
increase the difficulty of controlling these factors. 

1. Reactance. When human beings are observed, they react. 
Behavior is therefore affected by the act of being observed 
and thus the "natural" behavior is affected. The only 
practical solution is to "spy" on the subjects which raises 
a number of ethical questions. 

2. Contextual effects. Human behavior is a function of where 
they are located at the time of observation. To observe a 
subject in all of their "natural" settings in nearly 
impossible and the researcher must then select those of 
greatest importance. 

3. Distortion of the "natural" stream of behavior. To 
overcome contextual effects, some social scientists have 
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employed time-and-event sampling techniques to measure pre- 
determined behavioral categories in a number of different 
settings. This approach has been criticized on the grounds 
that it distorts the natural flow of events. The opposite 
approach, that of recording everything, is criticized for 
leading to mounds of unmanageable data. 

4, The internal/external distinction. The interaction 
between observable behavior, underlying mental processes and 
underlying physiological processes, and the ways of 
assessing and measuring each have implications on the kinds 
of observations to be made and recorded, 

5, The base line problem. While in the physical sciences, 
many measurements can be reduced to some quasi-absolute 
value, in the social sciences, this is harder to do. 
Behavior must be measured against a comparison group whose 
relevant characteristics are known. Cross-cultural studies 
attempt to extend the baseline beyond the confines of one 
society and the control conditions are difficult to achieve 
in thece designs, 

6, The relationship between variables over time. Many 
research questions in the social sciences are concerned with 
how two variables are related over time. Thus, to establish 
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relationship such as age and performance, the investigator 
may test several groups which differ according to age. To 
draw conclusions about how two or more variables co-va*.y, 
researchers must ensure that they include sufficient 
measurement points to their design and that these points are 
suitably spaced. 

7. Individual differences. Variance in human subjects is 
very great. Subjects will respond differently to 
theoretically identical stimuli depending on their age, sex, 
socioeconomic background, etc. This variance can be 
compounded exponentially in cross-cultural studies- The 
problem is to uncover how these differences systematically 
interact with the stimulus of j.nterest. A second challenge 
is treating individual differences as extraneous variables 
and concentrating on the "pure" effect of the stimulus, 
something which is only achievable with large N's, 
randomizing subjects to conditions, statistical juggling, 
etc. But sampling all of the cultures of the world in order 
to test the universal application of a hypothesis, is not 
feasible in most studies. 

As this list suggests, using observational methods in cross- 
cultural studies is extremely challenging under the best of 
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circumstances. In discussing the historical weaknesses of these 
kinds of studies, Bochner (1986) suggests that observational 
methods have tended to be used by social scientist who have 
employed less than rigorous methods in collecting data. Two 
general problem plague these kinds of efforts. 

1. The researcher is usually highly visible and this has an 
effect on the subjects being observed and their behavior. 
This effect (reactance) is heightened in cross-cultural 
contexts where observer and observed are culturally quite 
distinct. 

2. Because this kind of research is usually only 
incidentally interested in confirming hypotheses, the 
selection of subjects and their observations tend to be less 
standardized and therefore, less concerned with 
quantifiability and statistical testing. The results and 
conclusions tend to be based on the subjective 
interpretations of the investigator, especially when there 
are cross-cultural differences between the observer and the 
observed. 

Bochner (1986) suggests that these problems can be overcome 
by effective research strategy and an appropriate methodology. The 
research strategy should include an exploratory study that (a) 
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identifies the target behavior, (b) quantifies the target 
behavior, (c) uses sampling of subjects and the time periods of 
observation and (d) forms a clear hypothesis to be ter^ted. Through 
the confirmatory studies, the investigator wants to study and 
predict actual behavior in a natural or "real" setting. These 
purposes are best served by directly measuring these acts. Thus, 
systematic observation is called into service. 

To overcome the problem of reactance, Bochner (1986) 
discusses a methodology which he and others call the "unobtrusive" 
method (Webb et al. cited in Bochner 1986). This methodology aims 
at (a) selecting a suitable setting for the study, (b) selecting a 
suitable method for recruiting the subjects, (c) determining how 
the dependent variables will be defined and quantified, (d) 
devising a procedure for processing the subjects through the 
investigation at an efficient rate, and (e) recording responses 
without either the subjects or the general public becoming aware 
that there is an experiment being conducted. These techniques 
(along with an understanding of the problems they are attempting 
to overcome) are particularly important to those conducting 
comparative studies between cultures. 

Comparative Studies 
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The application of observational methods to behavioral 
science disciplines has resulted in a blending of the 
anthropologist's techniques with hypothesis testing. Comparative 
studies in the cross-cultural context are usually designed to 
extend the predictive range of a hypothesis already demonstrated 
in one culture. The researcher uses many individual observations 
to build a case for generalizing a social or psychological law or 
theory across many cultures (Lonner & Berry, 1988). This usually 
involves drawing conclusions from an "emic," intra-cultural study, 
and testing them for applicability in an "etic" or cross-cultural 
study. 

One problem associated with this type of comparative study 
is that of perception (referred to as the "emic-etic" or "insider- 
outsider" dilemma) (Smith & Tayeb, 1988; Brislin et al., 1973). 
When conducting an emic - intra-cultural - study, the researcher's 
insider orientation may permit him/her to interpret behavior in 
the same way the subject or group interprets it. When conducting 
etic - cross-cultural - studies, the researcher does not have the 
insider's advantage. Behaviors aay appear totally different to 
those performing then than to the researcher observing them. Smith 
& Tayeb (1988) attribute some of these differences to problems of 
translation and non-equivalence of measures. "But some differences 
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stem from the fact that members of a given culture share complex 
systems for decoding meaning in one another's behavior that may be 
inaccessible to outsiders" (p. 154). Th\is, an outsider may not be 
able to accurately observe or measure the behavior of the culture 
being studied, an obstacle which continues to challenge the cross- 
cultural investigator. 

Berry (as cited in Brislin et al., 1973) suggests the 
following procedure to combat the emic-etic dilemma. 

1. Study only those aspects of behavior that are 
functionally equivalent between cultures. 

2. Descriptive categories derived from past research, 
perhaps in only one culture, can then be applied to the 
behavior in the other culture under investigation. This is 
the imposed etic. 

3. The imposed descriptive categories must be adapted so 
that they explain the cultural practices of the culture from 
their point of view. This is the emic description. 

4. Shared categories can then be used to build up new 
categories valid for both cultures (derived etic) and can be 
expanded to other cultures until a universal is established. 

5. New instruments (\nd research techniques can be devised 
and validated using the derived etic as a conceptual base. 
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These instruments must, of course, be equivalent in meaning 
to the members of the cultures under study, {pp. 24-25) 
Empirical based correlational methods for eliciting the 
meaning of concepts in the cultures under study have been 
developed {Nowak, 1962; Przeworski & Teune, 1966; Berrien, 1970 as 
cited in Brislin et al., 1973). After a careful preliminary study, 
a set of questions can be formulated around some construct which 
has "identical" meaning in the cultures under consideration. Once 
these common questions are found, additional questions may be 
derived which are specific to each of the cultures. These 
questions are designed to produce information that is culturally 
specific. The answers to the culture specific questions are then 
correlated to the derived set of "identical" questions. 

Poortinga & Malpass (1986) describe a process of classifying 
cross-^cultural "universes" which can be used to define scales for 
the sake of comparing cultures. "For example, one could ask 
anthropologists and missionaries to draw up lists of the 100 most 
important plants and animals for any local region in the world and 
declare scales based on theses lists as adequate comparison 
scales" (p. 24). Comparison scales can be defined when all elements 
in the respective universe are known and either the entire 
universe or a representative sample is used for its measurement. 
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These techniques, while theoretically feasible, are 
tremendously difficult to carry out in practice. These 
difficulties tend to greatly limit the topics which can be studied 
and may call into question the conclusions derived from 
comparative studies. 

Experimental Approaches 
Cross-cultural research is subject to the same kinds of 
threats to internal validity as intra-cultural research. But 
because it is usually being conducted in field settings under 
"different" and less than ideal conditions, avoiding bias and 
reducing threats to validity may take much more effort than the 
same study in the researcher's own culture. In addition to these 
"known" factors r there may be unknown or unanticipated problems 
that arise due to the foreign context of the study. 

Campbell (as cited in Brislin et al., 1973) lists fifteen 
factors that may jeopardize the validity of any explanation of 
obtained data. Using the case example of a study on parent-child 
relationships, Brislin et al. (1973) list these factors with 
cross-cultural examples. 

1. History. Another societal variables, such a socioeconomic 
status, may affect questionnaire responses rather than the 
variable of parent-child relations in which the investigator 
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is interested* 

2. Testing. Members of some cultures have had more 
experience taking paper and pencil tests than others, and 
this experience might affect scores on tests taken 
subsequently* 

3. Maturation. Subjects in one culture may be more likely to 
become fatigued in a one-hour session than subjects in 
another. 

4. Instrumentation. Observers of different cultures may have 
response bias habits which might affect the data. 
Translation inadequacies constitute instrumentation errors. 

5. Statistical Regression. If cultures are selected on the 
basis of their extremity on a dimension, this might affect 
responses to a dependent variable that has less than a 
perfect correlation with the independent variable. 

6. Instability. This factor refers to fluctuations of 
observed responses due to unreliable tests or chance factors 
in samples. Statistical tests are properly applied to 
investigate this (but only this) factor. In cross-cultural 
work, as in all social science research, statistical tests 
are applied to see if obtained results can be interpreted as 
being not due to such chance fluctuations ♦ 
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7. Selection. The subjects an experimenter selects may be 
chosen because they are more cooperative and talkative. The 
cultures selected may be chosen because of convenience 
rather than scientific interest. 

8. Experimental mortality. This factor refers to "losing" 
subjects or respondents. Subjects in different cultures may 
be differentially difficult to schedule for a second 
interview. 

9. Selection-maturation or selection-testing interaction. 
This more complex factor is a possible alternative 
explanation to the observation that members of one culture 
increase in amount learned over time, while those of another 
culture do not. It could be that members of one of the 
selected cultures were more likely to learn because of their 
past experience with similar situations or because they were 
more likely to mature naturally during the course of the 
experiment . 

10. Interaction effects of testing. A pretest given to 
member of some cultures may sensitize them so that they feel 
they know bow to respond to the experimental variable. This 
sensitization, because of the pretest, may be less or 
greater in other cultures. 
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11. Interaction of selection and experimental treatment. The 
selection biases, discussed in number seven above, may yield 
a sample of very cooperative subjects in one culture but 
suspicious subjects in another. They may then respond 
differentially to the experimental treatment, thus making 
generalizations and data interpretation difficult. 

12. Reactive effects. This factor refers to the "I'm being 
studied-how should I act?" phenomenon (as discussed^ in the 
section on observational methodologies). 

13. Multiple-treatment interference. If subjects are in an 
experiment or testing session with more than one part, the 
first part might effect the second, and thus generalization 
to the responses of people in another culture to only the 
second part becomes impossible. 

14. Irrelevant responsiveness of measures. A questionnaire 
given in several cultures may contain items meaningful in 
one culture but strange or meaningless to members of 
another. 

15. Irrelevant replicability of treatments. In an 
experimental treatment with many variables (e.g., a ten-hour 
reading session which includes pleasant teacher, television, 
and culture-related teaching devices) , a replication in 
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another culture may not have the same effect because the key 
variable that caused the original effect may be 
inadvertently omitted or is less salient to members of the 
second culture, (pp. 17-19) 

This list of possible threats to validity sets the stage for 
the major difficulty in interpreting results of cross cultural 
research; the elimination of alternative explanations or "rival 
hypothesis" in a given set of outcomes (Poortinga & Malpas, 1986; 
Brislin et al., 1973). At every stage of the study, the researcher 
should be aware of and list potential reasons that could explain 
data in addition to the preferred hypothesis being tested. These 
explanations can be substantive (another cultural difference) or 
methodological. 

There is no technique for eliminating these rival hypothesis 
wholesale. They must be approached one-by-one and eliminated 
through the introduction of supplementary (systematic) variation 
within one or both of the comparison populations. This approach 
calls for designing research in such a way that the problem is 
addressed through several dimensions and the findings in each of 
these correlated with each other to substantiate (or call into 
question) the preferred hypothesis. 
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The designer of a carefully planned study should be able to 
rule out all threats so as to support the preferred interpretation 
of cultural differences. If a researcher does this, he/she should 
be able to answer questions that are raised about the 
interpretation of the results. But in order to do this adequately, 
several sets of data will probably need to be gathered. 

Sampling 

As in all studies, sampling is a critical design 
consideration. Plausible rival hypothesis are often suggested by 
examining the sampling plan of a piece of research. In cross- 
cultural studies, rival hypothesis are always suggested since the 
differences found may be attributed to dissimilar selection 
methods or disparate qualities of samples (ie. age, socioeconomic 
status, etc.). In much cross-cultural research there is no 
sampling plan; an experimenter is likely to use people who are 
talkative, cooperative or who are readily available. Even when 
random or stratified samples are attempted, the necessary 
demographic information may not be available and technical 
problems of not-at-homes or differential refusal rates may plague 
interpretations (Brislin et al., 1973). 

Random sampling is rare in cross-cultural studies. Vhen it 
is undertaken, the costs are great. Even when methodological 
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problems are overcome, the random samples themselves may be 
unequal in terms of socioeconomic or other cultural factors. Thus 
rival hypotheses are suggested even when random sampling is used. 
This does not mean that random sampling is worthless* It may be 
valuable in eliminating some kinds of systematic bias such as when 
only cooperative people are used. But whether sampling is random 
or non-random, |he alternative hypotheses will have to be examined 
and eliminated. 

In discussing cross-cultural random sampling, Lonner & Berry 
(1986) mention three general "rules." The first is that a 
representative sample may only be possible when the population is 
extremely homogeneous and there exists some accurate and up-to- 
date central registry that can be used as a sampling framework, 
something which is usually unavailable in developing nations. The 
second is that the best samples in cross-cultural research are 
those that result from the most careful attempts that the 
circumstances permit to approximate the kind of sample needed to 
permit the proper execution of the research. The use of non-random 
sampling in cross-cultural research is almost a given due to 
"indefinite populations, unavailable sampling frames, small 
budgets, lack of time, inexperienced personnel, pressure for 
results, and the like" (Warwick as cited in Lonner & Berry, 1986). 
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In stating their third sampling rule, Lonner & Berry first 
list a number of non-random sampling techniques found in the 
literature. 

1. Samples of convenience. Individuals are taken from some 
convenient collectively such as a school or factory. Their 
accessibility makes them a very cost-effective sample. This 
technique departs to a degree from true representativeness. 
Because flux of time and situation will alter the 
configuration of such samples, they are not replicable. 

2. Bunch or grab samples, also called fortuitous or 
haphazard samples. Individuals are selected in bunches 
because they are convenient and fit into the research 
scheme. Representativeness is a secondary consideration in 
this kind of sampling. 

3. Judgmental sampling, also called deliberate, purposive or 
selective sampling. In this case, the researcher uses 
judgement in selecting individuals who are instrumental in 
gathering certain kinds of data or testing certain 
hypothesis. 

4. Expert choice samples. This kind of sampling is similar 
to judgmental sampling but uses an "expert" instead of the 
researcher to select the sample. 
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While recognizing that random sampling is the ideal method, any of 
these alternative non-random sampling techniques may be the best 
approach possible under the circumstances • But when they are used, 
rule number three comes into effect which states that **Vhen 
circumstances warrant departure from random sampling procedures, 
be as precise as possible in detailing the procedures you used, 
and make special note of any factor (s) which influenced the 
present sampling procedure but may not affect future procedure^** 
(pp. 88) . 

Although sampling xs always of critical interest when 
considering validity issues, it is apparent that cross-cultural 
research is faced with some special challenges which can not 
always produce the advantages associated with random samples. 
Assuming methodological problems can be overcome, cultural factors 
may render the samples less than equivalent. If the three rules 
described above are followed, however, the best sampling procedure 
under the circumstances should emerge and its rationale be 
described for analysis by others. While non-random sampling may 
limit the generalizability of the- results, compensating for 
alternative hypotheses raised by the sampling plan will greatly 
strengthen the study. 

Instrumentation 
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Social and behavioral scientists often use survey 
instruments to collect data. When conducting cross-cultural 
studies, these instruments are either (a) "standard" 
questionnaires developed in one culture (usually an industrialized 
nation) and used in another, or (b) an instrument developed by the 
researcher for a specific cross-cultural study. In either case, 
the instrument may need to be translated. Translated instruments 
run the risk of item non-equivalence in comparative studies or of 
item inappropriateness when standard scales, developed in another 
context, are used. 

The use of standard questionnaires have the advantage that 
they are already developed (saving time and money) and that they 
have an established track record, thus increasing their 
respectability. Disadvantages include the fact that they have been 
developed in one cultural matrix and may miss important aspects of 
a phenomenon as seen by people in other cultures, and that their 
use also encourages intellectual timidity in researchers who might 
otherwise make a more original contribution. Thus the creation of 
new instruments, or the modification of existing ones to meet 
cultural requirements of the target group is suggested (Brislin, 
1986; Segall, 1986). 
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Brislin (1973; 1980; 1986) has been instrumental in 
establishing guidelines for writing questions for use in cross- 
cultural situations. These guidelines are intended to assure that 
translators will: 

1. Have a clear understanding of the original language item. 

2. Have a high probability of finding a readily available 
target language equivalent so that they do not have to use 
convoluted or unfamiliar terms. 

3. Be able to produce target language items readily 
understandable by the eventual set of respondents who are 
part of the data-gathering stage of the research project. 
These guidelines are aimed at producing translatable English 

and are applicable to new item production or item modification. 

1. Use short simple sentences of less than sixteen words. 

2. Employ the active rather than passive voice. 

3. Repeat nouns instead of using pronouns. 

4. Avoid metaphors and colloquialisms. 

5. Avoid the subjunctive, for example verb forms with 
"could." "would," or "should." 

6. Add sentences to provide context for key ideas. 

7. Avoid adverbs and propositions telling "where" or "when." 

8. Avoid possessive forms where possible. 
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9. Use specific rather than general terms. 

10. Avoid words indicating vagueness regarding some event or 
thing (eg. probably, maybe, perhaps). 

• 11. Use wording familiar to translators- 

12. Avoid sentences with two different verbs if the verbs 
suggest two different actions, (pp. 144-149) 
The use of translatable English in item modification or the 
creation of new items and questionnaires is an important aspect of 
cross-cultural instrumentation. While focusing on this technical 
aspect of instrument development, it is important to mention that 
standard procedures for assuring external validity and reliability 
need to be followed as well. Otherwise, the instrument may be 
translatable but relatively worthless as a research tool. While 
the focus here is on producing English language instruments as the 
original medium, this should not be interpreted as an attempt to 
exalt English as the research language. It is important to 
recognize, however, that English is the most internationalized 
language today and that most researchers around the world will be 
able to read and utilize the instrument if it is developed in this 
language. It is also more feasible to translate from English to 
almost any other language or dialect in the world than to work 
from another language. 
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When creating new items for inclusion in new or old 
instruments, Brislin (1986) suggests that thorough library 
research first be undertaken in studying the target culture. The 
writings of anthropologists, sociologists, political scientists, 
missionaries and government officials should be sought. Another 
excellent library source of cross-cultural information is the 
Human Relations Area Files. These files contain documents 
catalogued by culture, author, author *s discipline and also by 
concepts covered. 

Brislin also suggest working with bi-lingual persons in item 
development. These people are conversant in both cultures and can 
often conceptualize items in both languages. By seesawing between 
the two cultures, the bi-lingual may be able to write equivalent 
items in both languages. He/she may also have important insights 
into possible response bias in the target culture and help to 
formulate questions in such a way 'that this hazard is avoided. 

The concept of moving back and forth between cultures is key 
to any translation effort. This technique known as back- 
translation or decentering, uses translators from both languages 
to blindly translate back and forth their counterpart's work. 
Through repeated translations and back translations, equivalent 
items "survive" and can be considered "etic" since words were 
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found in both languages to conceptualize them. The procedure can 
be depicted graphically as follows. 

source to target to source to target to source 
\ /\ /\ /\ I' 

\ I \ I \ I \ I 

\ I \ I \ I \ I 

\ I \ I \ / \ / 

\/ \/ \/ \/ 

translator II translator #2 translator 13 translator #4 

Back translation is a very useful technique in cross- 
cultural instrument development, but as with any other instrument, 
the final product should be pretested to assure its usefulness in 
the context in which it will be employed. The main advantage to 
back translation is that it allows researchers a degree of control 
over the instrument development since they can compare the 
original and back translations and make inferences about the 
quality of the translation. Even when researchers are conversant 
in the target language, Brislin suggest a similar process be used 
with monolinguals. The researcher can write the original in the 
target language, take it through a series of monolingual rewrites, 
and then compare the final version to the original. 

While on the subject of instrument development, Segall 
(1986), in a discussion on the design of cross-cultural attitude 
scales, mentions the importance of balancing the items between 
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those worded positively and those worded negatively. The item 
"Mexicans are warm and honest people," might be balances with one 
like "Mexicans are likely to try to cheat outsiders." On a Likert 
scale, answers to these two questions should be somewhat opposed, 
that is, if both are answered as "strongly agree," this might 
reveal a bias towards agreeing with all items. 

Segall (1986) also mentions an interesting technique for 
scale analysis which he refers to as the "Guttman analysis." 

Create an a priori set of items, worded in both directions, 
administer them to a sample of respondents, arrange 
respondents in order and items in order, eliminate those 
items which persistently contribute disorder to the array, 
and, finally, live with reduced set of items as the best 
possible scale. Respondents are then re-scored on that best 
possible scale and the new scores constitute the best 
\ possible estimate of the respondents* attitudes, (p. 284) 
This particular technique suggests itself in the absence of 
adequate prior reliability testing or as a double check on the 
accuracy of that process. 

Once the instrument has been designed, translated and 
properly tested, its administration poses other problems the 
researcher should be aware of. The interview or questionnaire 
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procedure may be subject to respondent biases which have been 
creatively named by Pareek & Rao (1980). 

1. The rudeness bias. Researchers forget that they are 
requesting valuable time of people in other cultures and 
plunge ahead without adequate knowledge of local norms and 
protocol. 

2. The sucker bias. Respondents give nonsensical answers in 
the spirit of fun to see how seriously the researcher will 
take their silliness. 

3. The I-can-answer-any-question bias. The deep-seated 
conviction by some respondents that all questions should be 
answered whether or not they have the knowledge or 
information asked for. 

4. The courtesy bias. The respondents try to discover what 
the researcher is looking for and accommodate their answers 
accordingly. 

5. The hidden premise bias. The respondents try to figure 
out what the researcher is really after. Government 
sponsored research is seen as an attempt to extend 
governmental control. 

There is no single solution to dealing with these potential 
sources of bias. As has been suggested, the researcher should be 



OK) 
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as knowledgeable as possible about the target culture and 
anticipate the kind of bias that may occur. Persons from the 
target group working on instrument development may also be able to 
anticipate some of these problems and guide the researcher in 
developing and adequate strategy. If the instrument is tested in 
an **real*' situation^ biases may be identified at that time and 
compensated for in the final administration. 

Summary 

Conducting research in a cross-cultural setting is 
attractive to scientists from many disciplines for a number of 
reasons. But as has been expressed, it is not simply a matter of 
"business-as-usual" in another setting. The researcher needs to 
deal with the "normal" sources of bias and threats to validity 
common to all research design and administration, but in cross- 
cultural research, these are greatly exacerbated by the foreign 
field setting. In addition, there are a number of methodological 
problem areas which are never encountered in intra-cultural 
studies. 

By providing an overview of possible sources of bias and 
threats to validity while conducting cross-cultural research, it 
is hoped that the cross-cultural researcher, particularly the 
novice, can avoid some of the pitfalls inherent in this activity. 
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While beyond the scope of this paper to present in-depth 
development of techniques and methods to overcome these 
difficulties, the brief outlines presented should provide an 
orientation to procedures used by experienced researchers and 
guide the reader to their sources. The challenges to overcome are 
great but careful attention to the issues addressed in this paper 
should produce a worthwhile and rewarding research effort. 
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